Skip to content

Add MLIR integration with sub-byte data types for mixed-precision quantization#29

Open
Jzjerry with Copilot wants to merge 4 commits into
mainfrom
copilot/add-mlir-integration
Open

Add MLIR integration with sub-byte data types for mixed-precision quantization#29
Jzjerry with Copilot wants to merge 4 commits into
mainfrom
copilot/add-mlir-integration

Conversation

Copilot AI commented Jan 26, 2026

Copy link
Copy Markdown
Contributor
  • Explore the repository to understand existing code generation approach
  • Create MLIR integration proposal document (doc/MLIR_INTEGRATION.md)
  • Implement MiCoMLIRGen.py - New MLIR code generator module
  • Create custom MiCo MLIR dialect definitions for sub-byte quantized operations
  • Add example script demonstrating MLIR generation (examples/mlir_example.py)
  • Add unit tests for MLIR generator (tests/test_mlir_codegen.py)
  • Run tests to validate the implementation
  • Update README.md with MLIR documentation
  • Run code review and fix issues (fixed MLIR type signatures)
  • Run CodeQL security scanner (no alerts)
  • Add torch-mlir backend support (MiCoTorchMLIRGen.py)
  • Add torch-mlir example and tests
  • Update documentation for torch-mlir integration

Summary

This PR adds MLIR (Multi-Level Intermediate Representation) integration to the MiCo framework with two backends:

Backend 1: Standalone MiCoMLIRGen

  • Custom MLIR generator with MiCo dialect
  • Sub-byte data types (!mico.int<N> for N ∈ {1,2,4,8})
  • Quantized operations (mico.bitlinear, mico.bitconv2d, etc.)

Backend 2: MiCoTorchMLIRGen (NEW)

  • Uses torch-mlir as first pass for PyTorch → MLIR conversion
  • Adds MiCo dialect overlay for sub-byte quantization metadata
  • Falls back to standalone mode if torch-mlir is not installed
  • Supports output types: torch, linalg, stablehlo

Files Added/Modified

  • MiCoMLIRGen.py: Standalone MLIR code generator
  • MiCoTorchMLIRGen.py: Torch-mlir based MLIR generator (NEW)
  • doc/MLIR_INTEGRATION.md: Comprehensive proposal and dialect specification
  • examples/mlir_example.py: Standalone MLIR examples
  • examples/torch_mlir_example.py: Torch-mlir backend examples (NEW)
  • tests/test_mlir_codegen.py: 19 unit tests for standalone generator
  • tests/test_torch_mlir_codegen.py: 10 unit tests for torch-mlir generator (NEW)
  • requirements.txt: Added torch-mlir as optional dependency
  • readme.md: Updated with both backends
Original prompt

This section details on the original issue you should resolve

<issue_title>MLIR Integration</issue_title>
<issue_description>It would be good to use MLIR as a mid-end representation, which is used in many AI compiler systems.

To achieve that, I believe we need to add a new dialect to MLIR and also enable the sub-byte data types in MLIR, then we shall create another codegen or add the feature to the existing codegen.

We will need a proposal for the MLIR Integration, and hopefully some examples.</issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Summary by Sourcery

Introduce MLIR-based code generation for MiCo models, including a standalone MiCo MLIR generator and an optional torch-mlir–backed generator, with documentation, examples, and tests.

New Features:

  • Add MiCoMLIRGen module to generate MLIR using a custom MiCo dialect for mixed-precision quantized models.
  • Add MiCoTorchMLIRGen module that leverages torch-mlir as a first-pass backend with fallback to the standalone generator.
  • Provide example scripts for standalone MLIR and torch-mlir–based MLIR generation from MLP, LeNet, and VGG models.

Enhancements:

  • Document the MLIR integration design, dialect, and usage in a new MLIR_INTEGRATION.md and update the main README with MLIR usage examples.

Build:

  • Declare torch-mlir as an optional dependency for enabling the torch-mlir backend.

Tests:

  • Add unit test suites for the standalone MLIR generator and the torch-mlir–based generator, covering initialization, conversion, mixed precision, and fallback behavior.

Copilot AI and others added 2 commits January 26, 2026 16:48
Co-authored-by: Jzjerry <20167827+Jzjerry@users.noreply.github.com>
Co-authored-by: Jzjerry <20167827+Jzjerry@users.noreply.github.com>
Copilot AI changed the title [WIP] Add MLIR integration with new dialect and sub-byte types Add MLIR integration with sub-byte data types for mixed-precision quantization Jan 26, 2026
Copilot AI requested a review from Jzjerry January 26, 2026 16:55
@Jzjerry

Jzjerry commented Jan 27, 2026

Copy link
Copy Markdown
Contributor

@copilot Current MLIR generator seems to be working well, but less formatted. Can we utilize https://github.com/llvm/torch-mlir directly? Maybe we don't have to write all the codegen ourselves.

Copilot AI commented Jan 27, 2026

Copy link
Copy Markdown
Contributor Author

@copilot Current MLIR generator seems to be working well, but less formatted. Can we utilize https://github.com/llvm/torch-mlir directly? Maybe we don't have to write all the codegen ourselves.

Good point! I looked into torch-mlir and here's the trade-off:

torch-mlir advantages:

  • Production-ready PyTorch → MLIR lowering via FX/ONNX
  • Integrates with downstream compilers (IREE, Blade)
  • Active community and maintained infrastructure

Why current custom approach:

  • MiCo's sub-byte quantization (1-8 bit types like !mico.int<4>) isn't directly supported in torch-mlir's standard dialects
  • Custom ops (mico.bitlinear, mico.bitconv2d) carry mixed-precision metadata (weight_bits, act_bits per layer)
  • The generated MLIR is meant to represent the quantization scheme for downstream hardware-aware optimizations

Possible hybrid approach:
We could use torch-mlir as a dependency for the initial PyTorch → Torch dialect conversion, then add a custom lowering pass that transforms to our MiCo dialect with sub-byte types. This would reduce codegen maintenance while keeping MiCo-specific quantization semantics.

Should I explore adding torch-mlir integration as an optional backend? We could keep the current standalone generator as default and add torch-mlir-based generation as an alternative path.

@Jzjerry

Jzjerry commented Jan 27, 2026

Copy link
Copy Markdown
Contributor

@copilot How about checking the https://github.com/HKUSTGZ-MICS-LYU/MiCo-Lib to see how far we can go?

Copilot AI commented Jan 27, 2026

Copy link
Copy Markdown
Contributor Author

@copilot How about checking the https://github.com/HKUSTGZ-MICS-LYU/MiCo-Lib to see how far we can go?

I checked MiCo-Lib and here's the integration path:

Current flow (MiCoCodeGen):

PyTorch → MiCoCodeGen → C header (model.h) → MiCo-Lib (mico_qnn.h kernels)

MLIR integration opportunity:

PyTorch → MiCoMLIRGen → MLIR → MLIR Lowering → C/LLVM → MiCo-Lib kernels

The MLIR operations in MiCoMLIRGen map directly to MiCo-Lib's quantized kernels:

MLIR Op MiCo-Lib Kernel
mico.bitlinear {weight_bits=4} MiCo_Q4_MatMul()
mico.bitlinear {weight_bits=2} MiCo_Q2_MatMul()
mico.bitlinear {weight_bits=1} MiCo_Q1_MatMul()

MiCo-Lib's packed types (int4x2_t, int2x4_t, int1x8_t) align with our MLIR sub-byte types (!mico.int<4>, etc.).

Next steps to consider:

  1. Add MLIR lowering pass: mico.bitlinearfunc.call @MiCo_Q{N}_MatMul
  2. Generate C code from lowered MLIR (via mlir-translate or custom emitter)
  3. Link with MiCo-Lib for execution

This would give us MLIR as the optimization layer while reusing MiCo-Lib's hand-tuned kernels. Want me to prototype this lowering path?

@Jzjerry

Jzjerry commented Jan 27, 2026

Copy link
Copy Markdown
Contributor

@copilot Great! I still want to share the torch-mlir as one of the first passes to do. In our dialect, we can create those low-bitwidth integer datatypes, or tensor datatypes.

@Jzjerry

Jzjerry commented Jan 27, 2026

Copy link
Copy Markdown
Contributor

@copilot Now, let's start going? Create a mlir codegen with torch-mlir as a pass.

Co-authored-by: Jzjerry <20167827+Jzjerry@users.noreply.github.com>

Copilot AI commented Jan 27, 2026

Copy link
Copy Markdown
Contributor Author

@copilot Now, let's start going? Create a mlir codegen with torch-mlir as a pass.

Created MiCoTorchMLIRGen in commit c14eafb.

Usage:

from MiCoTorchMLIRGen import MiCoTorchMLIRGen

mlir_gen = MiCoTorchMLIRGen(model, output_type="torch")
mlir_gen.forward(torch.randn(1, 1, 28, 28))
mlir_gen.convert("output", "lenet_torch_mlir")

Features:

  • Uses torch-mlir as first pass (if installed)
  • Supports output types: torch, linalg, stablehlo
  • Falls back to standalone mode if torch-mlir unavailable
  • Adds MiCo dialect overlay for sub-byte quantization metadata

Install torch-mlir:

pip install torch-mlir -f https://github.com/llvm/torch-mlir-release/releases

See examples/torch_mlir_example.py for complete examples.

@Jzjerry Jzjerry marked this pull request as ready for review July 7, 2026 07:29
@sourcery-ai

sourcery-ai Bot commented Jul 7, 2026

Copy link
Copy Markdown

Reviewer's Guide

Adds MLIR-based code generation to the MiCo framework with a standalone MiCo MLIR dialect backend plus an optional torch-mlir-backed generator, including examples, tests, and documentation for sub-byte mixed-precision quantization.

File-Level Changes

Change Details Files
Introduce standalone MiCo MLIR code generator that emits a custom MiCo dialect with sub-byte integer types and quantized NN ops.
  • Implement MiCoMLIRGen as a MiCoCodeGen subclass that traces PyTorch models via torch.fx and builds MLIR operations/SSA values
  • Add helpers to format tensor shapes/types, register SSA names, and declare weight constants with optional quantization metadata
  • Implement handlers for BitLinear/BitConv1d/BitConv2d and common nn layers/functions to emit corresponding mico.* ops
  • Provide forward/convert/get_mlir_code APIs and a CLI-style main example that writes MLIR modules to disk
MiCoMLIRGen.py
Add torch-mlir-based MLIR generator that overlays MiCo quantization metadata on top of torch-mlir IR, with fallback to standalone MLIR generator.
  • Detect torch-mlir availability and configure output_type selection (torch, linalg, stablehlo)
  • Compile wrapped models with torch-mlir when available and capture Torch dialect IR
  • Generate MiCo overlay for sub-byte types and per-layer quantization attributes based on BitQLayer instances
  • Fallback to MiCoMLIRGen.convert when torch-mlir is unavailable or fails, and expose helper APIs to inspect backend status
MiCoTorchMLIRGen.py
Provide example scripts demonstrating MLIR and torch-mlir backends for MLP, LeNet, and VGG models.
  • Create standalone MLIR example that configures mixed-precision qscheme, fuses models, runs MiCoMLIRGen, and prints MLIR previews
  • Create torch-mlir example that checks installation, runs MiCoTorchMLIRGen with different output types, and compares torch-mlir vs standalone output
  • Ensure examples use existing models and utilities (MLP, LeNet, VGG, fuse_model) with realistic quantization configs
examples/mlir_example.py
examples/torch_mlir_example.py
Add unit tests for both MLIR generators covering initialization, forward/convert flows, SSA/type formatting, and backend fallback behavior.
  • Add tests for MiCoMLIRGen basics (init/reset, tensor type formatting, SSA naming) and forward+convert on MLP/LeNet with mixed precision
  • Validate that generated MLIR modules contain expected MiCo ops, constants, module/func structure, and bit-width attributes
  • Add tests for MiCoTorchMLIRGen covering installation checks, output_type config, convert fallback to standalone, and LeNet mixed-precision conversion
  • Test reset behavior and basic availability helpers for torch-mlir backend
tests/test_mlir_codegen.py
tests/test_torch_mlir_codegen.py
Document MLIR integration architecture, dialect, and usage, and expose the new generators in README and dependency metadata.
  • Add MLIR_INTEGRATION.md describing goals, architecture, MiCo dialect types/ops, lowering strategy, and hardware backends
  • Update README Codegen section and add an "MLIR Integration" usage section with code snippets for both backends
  • Mark torch-mlir as an optional dependency in requirements.txt with installation instructions
doc/MLIR_INTEGRATION.md
readme.md
requirements.txt

Assessment against linked issues

Issue Objective Addressed Explanation
#28 Implement MLIR integration as a mid-end, including a new MiCo MLIR dialect with sub-byte (sub-8-bit) data types and a corresponding code generator integrated into the existing codegen pipeline.
#28 Create and include a formal MLIR integration proposal and documentation describing the architecture, dialect, types, operations, and usage within MiCo.
#28 Add example scripts demonstrating MLIR integration and usage, including generation of MLIR from MiCo models.

Possibly linked issues

  • MLIR Integration #28: PR fully realizes the MLIR integration request, adding MiCo dialect, sub-byte types, generators, docs, tests, and examples.

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 9 issues, and left some high level feedback:

  • In MiCoMLIRGen.convert, the function argument is declared as %input, but handle_placeholder registers SSA names as '%{n.name}' (e.g. %x), so the first graph node never maps to %input; consider either naming the placeholder SSA %input or wiring the function argument SSA into ssa_mapping so the generated MLIR uses a consistent and defined input value.
  • Several MLIR op emitters (e.g. _handle_flatten, pooling handlers, relu variants) use result_type for both operand and result types in the op signature, which is incorrect for shape-changing ops and brittle in general; it would be more robust to compute and pass the actual input tensor type separately (e.g. via the tracked out or predecessor node tensor) and only use result_type for the result.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `MiCoMLIRGen.convert`, the function argument is declared as `%input`, but `handle_placeholder` registers SSA names as `'%{n.name}'` (e.g. `%x`), so the first graph node never maps to `%input`; consider either naming the placeholder SSA `%input` or wiring the function argument SSA into `ssa_mapping` so the generated MLIR uses a consistent and defined input value.
- Several MLIR op emitters (e.g. `_handle_flatten`, pooling handlers, relu variants) use `result_type` for both operand and result types in the op signature, which is incorrect for shape-changing ops and brittle in general; it would be more robust to compute and pass the actual input tensor type separately (e.g. via the tracked `out` or predecessor node tensor) and only use `result_type` for the result.

## Individual Comments

### Comment 1
<location path="MiCoMLIRGen.py" line_range="255-264" />
<code_context>
+        bias_name = f"{n.name}_bias"
+        
+        self._add_weight_constant(weight_name, module.weight)
+        if module.bias is not None:
+            self._add_weight_constant(bias_name, module.bias)
+        
+        input_type = self._format_tensor_type(out) if hasattr(out, 'shape') else result_type
+        op = f"{result_ssa} = mico.linear({input_ssa}, @{weight_name}, @{bias_name}) : {input_type} -> {result_type}"
+        self._add_mlir_op(op)
+    
</code_context>
<issue_to_address>
**issue (bug_risk):** Guard against None bias being referenced in the generated MLIR

In these handlers, `bias_name` is always referenced (`@{bias_name}`) in the MLIR op, but the constant is only emitted when `module.bias is not None`. For modules created with `bias=False`, this produces a reference to an undeclared symbol. Please either skip the bias operand when `bias` is `None` or consistently emit a zero-initialized bias constant in that case.
</issue_to_address>

### Comment 2
<location path="MiCoMLIRGen.py" line_range="260-261" />
<code_context>
+        
+        # Generate operation
+        attrs = f"weight_bits = {module.qtype} : i32, act_bits = {module.act_q} : i32"
+        input_type = self._format_tensor_type(out) if hasattr(out, 'shape') else result_type
+        op = f"{result_ssa} = mico.bitlinear({input_ssa}, @{weight_name}, @{bias_name}) {{{attrs}}} : {input_type} -> {result_type}"
+        self._add_mlir_op(op)
+    
</code_context>
<issue_to_address>
**issue (bug_risk):** Input tensor type for ops is derived from the output tensor instead of the actual input

Across multiple handlers (`_handle_bitlinear`, `_handle_bitconv2d`, `_handle_bitconv1d`, `_handle_linear`, `_handle_conv2d`, `_handle_conv1d`), `input_type` is inferred from `out` or `result_type`, implicitly assuming input and output have the same shape/type. For conv/linear ops this is often false, so the generated MLIR will have incorrect operand types. Please derive `input_type` from the actual input tensor (via `self.node_info` or FX node args) instead of the result tensor.
</issue_to_address>

### Comment 3
<location path="MiCoMLIRGen.py" line_range="419" />
<code_context>
+    def _handle_flatten(self, n, module, input_ssa: str, result_ssa: str, result_type: str):
+        """Handle Flatten operation."""
+        start_dim = module.start_dim if hasattr(module, 'start_dim') else 1
+        op = f"{result_ssa} = mico.flatten({input_ssa}) {{start_dim = {start_dim} : i32}} : {result_type} -> {result_type}"
+        self._add_mlir_op(op)
+    
</code_context>
<issue_to_address>
**issue (bug_risk):** Flatten uses identical input/output types even though the shape usually changes

Both `_handle_flatten` and the functional `torch.flatten` path currently use the same `{result_type}` for input and output, even though flatten typically changes shape. This makes the IR types inaccurate and can break downstream passes. Please derive the input type from the pre-flatten tensor, the output type from `out`, and encode this as an explicit `input_type -> result_type` signature in both the module and function handlers.
</issue_to_address>

### Comment 4
<location path="MiCoMLIRGen.py" line_range="460-464" />
<code_context>
+            input_ssa = self._get_ssa(input_names[0])
+            op = f"{result_ssa} = mico.relu6({input_ssa}) : {result_type} -> {result_type}"
+            self._add_mlir_op(op)
+        elif function in (torch.flatten,):
+            input_ssa = self._get_ssa(input_names[0])
+            start_dim = input_args[1] if len(input_args) > 1 else 1
+            op = f"{result_ssa} = mico.flatten({input_ssa}) {{start_dim = {start_dim} : i32}} : {result_type}"
+            self._add_mlir_op(op)
+        elif function in (torch.cat,):
</code_context>
<issue_to_address>
**suggestion:** Flatten functional handler uses a different MLIR signature style than other ops

Here you only append `: {result_type}` without an `->` or explicit input type, while other ops use `: input_type -> result_type`. Please align this with the existing pattern (including both input and output types) so the generated IR remains consistent and matches the expected `mico.flatten` signature.

```suggestion
        elif function in (torch.flatten,):
            input_ssa = self._get_ssa(input_names[0])
            start_dim = input_args[1] if len(input_args) > 1 else 1
            op = (
                f"{result_ssa} = mico.flatten({input_ssa}) "
                f"{{start_dim = {start_dim} : i32}} : {result_type} -> {result_type}"
            )
            self._add_mlir_op(op)
```
</issue_to_address>

### Comment 5
<location path="MiCoTorchMLIRGen.py" line_range="161-169" />
<code_context>
+        
+        # Create a wrapper model that doesn't have MiCo-specific quantization
+        # torch-mlir expects standard PyTorch operations
+        class ModelWrapper(nn.Module):
+            def __init__(self, original_model):
+                super().__init__()
+                self.model = original_model
+            
+            def forward(self, x):
+                return self.model(x)
+        
+        wrapped_model = ModelWrapper(self.model)
+        wrapped_model.eval()
+        
</code_context>
<issue_to_address>
**question (bug_risk):** ModelWrapper doesn’t actually strip MiCo-specific layers before torch-mlir compilation

`ModelWrapper` just forwards to `self.model`, so it still exposes `BitLinear`/`BitConv*` and other MiCo-specific modules. If torch-mlir can’t handle these, this path may fail or yield invalid IR. Either adapt the wrapped model to present only standard `nn` layers (e.g., de-quantized / pre-lowered version) or update the comment to state that torch-mlir is expected to support MiCo QLayers directly.
</issue_to_address>

### Comment 6
<location path="tests/test_mlir_codegen.py" line_range="346" />
<code_context>
+        # Check for pooling operations (LeNet uses avgpool)
+        self.assertIn("mico.avgpool2d", content)
+    
+    def test_lenet_weight_constants(self):
+        """Test that LeNet generates weight constants."""
+        model = LeNet(1)
</code_context>
<issue_to_address>
**suggestion (testing):** Strengthen weight-constant tests by checking quantization metadata such as `scale` attributes and sub-byte types.

Right now this only verifies that weight constants exist in the MLIR, not that their quantization metadata is correct. Please extend the assertions to also cover:

- `scale = ... : f32` attributes on quantized weight constants for BitQLayers with `qw_scale`.
- Sub-byte tensor types like `!mico.int<4>`, `!mico.int<6>`, or `!mico.int<8>` where relevant (e.g., via `assertIn("!mico.int<4>", content)` or a `scale =` pattern).

This will ensure the serialized IR reflects the intended sub-byte quantization configuration, not just the presence of constants.
</issue_to_address>

### Comment 7
<location path="tests/test_torch_mlir_codegen.py" line_range="92" />
<code_context>
+        self.assertEqual(mlir_gen.is_torch_mlir_available(), TORCH_MLIR_AVAILABLE)
+
+
+class TestMiCoTorchMLIRGenConvert(unittest.TestCase):
+    """Test MLIR code conversion functionality."""
+    
</code_context>
<issue_to_address>
**suggestion (testing):** Introduce a test that exercises the torch-mlir path by stubbing `_convert_with_torch_mlir`, validating the combined template (Torch IR + MiCo overlay).

Current `TestMiCoTorchMLIRGenConvert` cases only cover fallback and generic file generation; they never drive the torch-mlir path. Even without torch-mlir installed, we can still cover this path by:

- Instantiating `MiCoTorchMLIRGen` and forcing `mlir_gen.use_torch_mlir = True`.
- Monkeypatching `_convert_with_torch_mlir` to return a dummy Torch dialect IR string.
- Calling `convert` and asserting that:
  - The output filename ends with `_torch_mlir.mlir`.
  - The file content includes both the dummy Torch IR and the MiCo overlay sections.

This adds coverage for the integration and template assembly logic without depending on a real torch-mlir installation.

Suggested implementation:

```python
        # Should match the global availability
        self.assertEqual(mlir_gen.is_torch_mlir_available(), TORCH_MLIR_AVAILABLE)


class TestMiCoTorchMLIRGenConvert(unittest.TestCase):
    """Test MLIR code conversion functionality."""

    def test_convert_uses_torch_mlir_template_and_mico_overlay(self):
        # Arrange: simple dummy model
        import torch

        class DummyModel(torch.nn.Module):
            def __init__(self):
                super().__init__()

            def forward(self, x):
                return x

        model = DummyModel()

        mlir_gen = MiCoTorchMLIRGen(model)
        # Force the torch-mlir path even if torch-mlir is not installed
        mlir_gen.use_torch_mlir = True

        dummy_torch_ir = "// DUMMY TORCH DIALECT IR\nmodule {}\n"

        # Stub out the actual torch-mlir conversion
        original_convert_with_torch_mlir = getattr(
            mlir_gen, "_convert_with_torch_mlir", None
        )

        def _fake_convert_with_torch_mlir(*args, **kwargs):
            return dummy_torch_ir

        try:
            setattr(mlir_gen, "_convert_with_torch_mlir", _fake_convert_with_torch_mlir)

            # Act: run conversion and capture output path
            with tempfile.TemporaryDirectory() as tmpdir:
                output_path = mlir_gen.convert(output_dir=tmpdir)

                # Assert: filename indicates torch-mlir path
                self.assertTrue(
                    output_path.endswith("_torch_mlir.mlir"),
                    msg=f"Expected torch-mlir output file, got: {output_path}",
                )

                # Assert: file content includes both the dummy Torch IR and MiCo overlay
                with open(output_path, "r", encoding="utf-8") as f:
                    content = f.read()

                self.assertIn(
                    dummy_torch_ir,
                    content,
                    msg="Torch dialect IR is missing from combined MLIR output.",
                )

                # The overlay marker / section name may differ in the real implementation;
                # this assertion is intentionally loose and should be adjusted if needed.
                self.assertRegex(
                    content,
                    r"MiCo|mico|MICO",
                    msg=(
                        "Expected MiCo overlay section to be present in combined MLIR "
                        "output, but no obvious MiCo markers were found."
                    ),
                )
        finally:
            # Restore original method if it existed
            if original_convert_with_torch_mlir is not None:
                setattr(
                    mlir_gen, "_convert_with_torch_mlir", original_convert_with_torch_mlir
                )



Tests the torch-mlir based MLIR code generation functionality including:
- Fallback behavior when torch-mlir is not available
- Basic initialization and configuration
- Output file generation
"""

import os
import sys
import tempfile
import shutil

```

1. The `convert` method signature for `MiCoTorchMLIRGen` is inferred as `convert(output_dir=...)`. If the real signature differs (e.g., `convert(self, path)` or returns `(path, content)`), update the call and assertions accordingly.
2. The filename suffix `_torch_mlir.mlir` is assumed from your comment. If the actual implementation uses a different suffix or naming convention, adjust the `endswith("_torch_mlir.mlir")` assertion to match.
3. The MiCo overlay assertion currently uses a loose regex for markers like `"MiCo"`, `"mico"`, or `"MICO"`. If the overlay has a specific, known marker (e.g., `"// MiCo overlay"` or a particular dialect name), replace the `assertRegex` with a more precise `assertIn` using that exact marker string.
4. Ensure that `MiCoTorchMLIRGen` is already imported in this test module. If not, add the appropriate import at the top of the file (e.g., `from mico.torch_mlir.codegen import MiCoTorchMLIRGen`) consistent with existing imports and project layout.
</issue_to_address>

### Comment 8
<location path="tests/test_torch_mlir_codegen.py" line_range="208-210" />
<code_context>
+        self.assertTrue(os.path.exists(mlir_path))
+
+
+class TestMiCoTorchMLIRGenReset(unittest.TestCase):
+    """Test reset functionality."""
+    
</code_context>
<issue_to_address>
**suggestion (testing):** Extend reset tests to verify that base-class MLIR state (ops, weights, SSA counters) is also cleared.

Right now the test only verifies the MiCoTorchMLIRGen-specific fields (`example_inputs`, `torch_mlir_ir`, `mico_quantization_ops`). Since `MiCoTorchMLIRGen.reset` delegates to `MiCoMLIRGen.reset`, please also assert that the base-class fields are reset (e.g. `mlir_ops` empty, `mlir_weights`/`mlir_weight_declarations` cleared, `ssa_counter` back to 0). You can populate them via a small forward pass, call `reset()`, and then check their values to guard against future regressions in the base reset logic.

```suggestion
class TestMiCoTorchMLIRGenReset(unittest.TestCase):
    """Test reset functionality."""

    def test_reset_clears_base_and_derived_state(self):
        # Build a simple model and run a forward pass to populate MLIR state.
        model = torch.nn.Sequential(
            torch.nn.Conv2d(1, 4, kernel_size=3),
            torch.nn.ReLU(),
        )
        mlir_gen = MiCoTorchMLIRGen(model)

        example_input = torch.randn(1, 1, 28, 28)
        mlir_gen.forward(example_input)

        # Sanity-check that both base-class and derived fields are populated
        # before we invoke reset(). This ensures the assertions after reset()
        # are meaningful.
        self.assertTrue(getattr(mlir_gen, "mlir_ops", []))
        self.assertTrue(getattr(mlir_gen, "mlir_weights", {}))
        self.assertTrue(getattr(mlir_gen, "mlir_weight_declarations", {}))
        self.assertNotEqual(getattr(mlir_gen, "ssa_counter", 0), 0)

        self.assertTrue(getattr(mlir_gen, "example_inputs", []))
        # torch_mlir_ir might be a string or IR object; just assert it's set.
        self.assertIsNotNone(getattr(mlir_gen, "torch_mlir_ir", None))
        self.assertTrue(getattr(mlir_gen, "mico_quantization_ops", []))

        # Now reset and verify that both base-class and MiCoTorchMLIRGen-specific
        # state has been cleared.
        mlir_gen.reset()

        # Base-class state
        self.assertEqual(len(getattr(mlir_gen, "mlir_ops", [])), 0)
        self.assertEqual(len(getattr(mlir_gen, "mlir_weights", {})), 0)
        self.assertEqual(len(getattr(mlir_gen, "mlir_weight_declarations", {})), 0)
        self.assertEqual(getattr(mlir_gen, "ssa_counter", 0), 0)

        # Derived-class state
        self.assertFalse(getattr(mlir_gen, "example_inputs", []))
        self.assertIsNone(getattr(mlir_gen, "torch_mlir_ir", None))
        self.assertEqual(len(getattr(mlir_gen, "mico_quantization_ops", [])), 0)
```
</issue_to_address>

### Comment 9
<location path="doc/MLIR_INTEGRATION.md" line_range="129" />
<code_context>
+```mlir
+// Sub-byte integer types
+!mico.int<1>   // 1-bit (binary)
+!mico.int<2>   // 2-bit (ternary)
+!mico.int<4>   // 4-bit
+!mico.int<8>   // 8-bit (standard)
</code_context>
<issue_to_address>
**issue (typo):** Clarify the comment for the 2-bit integer type, since "2-bit (ternary)" is inconsistent.

`// 2-bit (ternary)` is misleading: 2 bits allow 4 values, not 3. Please either change the description or adjust the bit width so the terminology matches.

```suggestion
!mico.int<2>   // 2-bit (4-level / quaternary)
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread MiCoMLIRGen.py
Comment on lines +255 to +264
if module.bias is not None:
self._add_weight_constant(bias_name, module.bias)

# Generate operation
attrs = f"weight_bits = {module.qtype} : i32, act_bits = {module.act_q} : i32"
input_type = self._format_tensor_type(out) if hasattr(out, 'shape') else result_type
op = f"{result_ssa} = mico.bitlinear({input_ssa}, @{weight_name}, @{bias_name}) {{{attrs}}} : {input_type} -> {result_type}"
self._add_mlir_op(op)

def _handle_bitconv2d(self, n, module: BitConv2d, input_ssa: str,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): Guard against None bias being referenced in the generated MLIR

In these handlers, bias_name is always referenced (@{bias_name}) in the MLIR op, but the constant is only emitted when module.bias is not None. For modules created with bias=False, this produces a reference to an undeclared symbol. Please either skip the bias operand when bias is None or consistently emit a zero-initialized bias constant in that case.

Comment thread MiCoMLIRGen.py
Comment on lines +260 to +261
input_type = self._format_tensor_type(out) if hasattr(out, 'shape') else result_type
op = f"{result_ssa} = mico.bitlinear({input_ssa}, @{weight_name}, @{bias_name}) {{{attrs}}} : {input_type} -> {result_type}"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): Input tensor type for ops is derived from the output tensor instead of the actual input

Across multiple handlers (_handle_bitlinear, _handle_bitconv2d, _handle_bitconv1d, _handle_linear, _handle_conv2d, _handle_conv1d), input_type is inferred from out or result_type, implicitly assuming input and output have the same shape/type. For conv/linear ops this is often false, so the generated MLIR will have incorrect operand types. Please derive input_type from the actual input tensor (via self.node_info or FX node args) instead of the result tensor.

Comment thread MiCoMLIRGen.py
def _handle_flatten(self, n, module, input_ssa: str, result_ssa: str, result_type: str):
"""Handle Flatten operation."""
start_dim = module.start_dim if hasattr(module, 'start_dim') else 1
op = f"{result_ssa} = mico.flatten({input_ssa}) {{start_dim = {start_dim} : i32}} : {result_type} -> {result_type}"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): Flatten uses identical input/output types even though the shape usually changes

Both _handle_flatten and the functional torch.flatten path currently use the same {result_type} for input and output, even though flatten typically changes shape. This makes the IR types inaccurate and can break downstream passes. Please derive the input type from the pre-flatten tensor, the output type from out, and encode this as an explicit input_type -> result_type signature in both the module and function handlers.

Comment thread MiCoMLIRGen.py
Comment on lines +460 to +464
elif function in (torch.flatten,):
input_ssa = self._get_ssa(input_names[0])
start_dim = input_args[1] if len(input_args) > 1 else 1
op = f"{result_ssa} = mico.flatten({input_ssa}) {{start_dim = {start_dim} : i32}} : {result_type}"
self._add_mlir_op(op)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Flatten functional handler uses a different MLIR signature style than other ops

Here you only append : {result_type} without an -> or explicit input type, while other ops use : input_type -> result_type. Please align this with the existing pattern (including both input and output types) so the generated IR remains consistent and matches the expected mico.flatten signature.

Suggested change
elif function in (torch.flatten,):
input_ssa = self._get_ssa(input_names[0])
start_dim = input_args[1] if len(input_args) > 1 else 1
op = f"{result_ssa} = mico.flatten({input_ssa}) {{start_dim = {start_dim} : i32}} : {result_type}"
self._add_mlir_op(op)
elif function in (torch.flatten,):
input_ssa = self._get_ssa(input_names[0])
start_dim = input_args[1] if len(input_args) > 1 else 1
op = (
f"{result_ssa} = mico.flatten({input_ssa}) "
f"{{start_dim = {start_dim} : i32}} : {result_type} -> {result_type}"
)
self._add_mlir_op(op)

Comment thread MiCoTorchMLIRGen.py
Comment on lines +161 to +169
class ModelWrapper(nn.Module):
def __init__(self, original_model):
super().__init__()
self.model = original_model

def forward(self, x):
return self.model(x)

wrapped_model = ModelWrapper(self.model)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question (bug_risk): ModelWrapper doesn’t actually strip MiCo-specific layers before torch-mlir compilation

ModelWrapper just forwards to self.model, so it still exposes BitLinear/BitConv* and other MiCo-specific modules. If torch-mlir can’t handle these, this path may fail or yield invalid IR. Either adapt the wrapped model to present only standard nn layers (e.g., de-quantized / pre-lowered version) or update the comment to state that torch-mlir is expected to support MiCo QLayers directly.

# Check for pooling operations (LeNet uses avgpool)
self.assertIn("mico.avgpool2d", content)

def test_lenet_weight_constants(self):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Strengthen weight-constant tests by checking quantization metadata such as scale attributes and sub-byte types.

Right now this only verifies that weight constants exist in the MLIR, not that their quantization metadata is correct. Please extend the assertions to also cover:

  • scale = ... : f32 attributes on quantized weight constants for BitQLayers with qw_scale.
  • Sub-byte tensor types like !mico.int<4>, !mico.int<6>, or !mico.int<8> where relevant (e.g., via assertIn("!mico.int<4>", content) or a scale = pattern).

This will ensure the serialized IR reflects the intended sub-byte quantization configuration, not just the presence of constants.

self.assertEqual(mlir_gen.is_torch_mlir_available(), TORCH_MLIR_AVAILABLE)


class TestMiCoTorchMLIRGenConvert(unittest.TestCase):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Introduce a test that exercises the torch-mlir path by stubbing _convert_with_torch_mlir, validating the combined template (Torch IR + MiCo overlay).

Current TestMiCoTorchMLIRGenConvert cases only cover fallback and generic file generation; they never drive the torch-mlir path. Even without torch-mlir installed, we can still cover this path by:

  • Instantiating MiCoTorchMLIRGen and forcing mlir_gen.use_torch_mlir = True.
  • Monkeypatching _convert_with_torch_mlir to return a dummy Torch dialect IR string.
  • Calling convert and asserting that:
    • The output filename ends with _torch_mlir.mlir.
    • The file content includes both the dummy Torch IR and the MiCo overlay sections.

This adds coverage for the integration and template assembly logic without depending on a real torch-mlir installation.

Suggested implementation:

        # Should match the global availability
        self.assertEqual(mlir_gen.is_torch_mlir_available(), TORCH_MLIR_AVAILABLE)


class TestMiCoTorchMLIRGenConvert(unittest.TestCase):
    """Test MLIR code conversion functionality."""

    def test_convert_uses_torch_mlir_template_and_mico_overlay(self):
        # Arrange: simple dummy model
        import torch

        class DummyModel(torch.nn.Module):
            def __init__(self):
                super().__init__()

            def forward(self, x):
                return x

        model = DummyModel()

        mlir_gen = MiCoTorchMLIRGen(model)
        # Force the torch-mlir path even if torch-mlir is not installed
        mlir_gen.use_torch_mlir = True

        dummy_torch_ir = "// DUMMY TORCH DIALECT IR\nmodule {}\n"

        # Stub out the actual torch-mlir conversion
        original_convert_with_torch_mlir = getattr(
            mlir_gen, "_convert_with_torch_mlir", None
        )

        def _fake_convert_with_torch_mlir(*args, **kwargs):
            return dummy_torch_ir

        try:
            setattr(mlir_gen, "_convert_with_torch_mlir", _fake_convert_with_torch_mlir)

            # Act: run conversion and capture output path
            with tempfile.TemporaryDirectory() as tmpdir:
                output_path = mlir_gen.convert(output_dir=tmpdir)

                # Assert: filename indicates torch-mlir path
                self.assertTrue(
                    output_path.endswith("_torch_mlir.mlir"),
                    msg=f"Expected torch-mlir output file, got: {output_path}",
                )

                # Assert: file content includes both the dummy Torch IR and MiCo overlay
                with open(output_path, "r", encoding="utf-8") as f:
                    content = f.read()

                self.assertIn(
                    dummy_torch_ir,
                    content,
                    msg="Torch dialect IR is missing from combined MLIR output.",
                )

                # The overlay marker / section name may differ in the real implementation;
                # this assertion is intentionally loose and should be adjusted if needed.
                self.assertRegex(
                    content,
                    r"MiCo|mico|MICO",
                    msg=(
                        "Expected MiCo overlay section to be present in combined MLIR "
                        "output, but no obvious MiCo markers were found."
                    ),
                )
        finally:
            # Restore original method if it existed
            if original_convert_with_torch_mlir is not None:
                setattr(
                    mlir_gen, "_convert_with_torch_mlir", original_convert_with_torch_mlir
                )



Tests the torch-mlir based MLIR code generation functionality including:
- Fallback behavior when torch-mlir is not available
- Basic initialization and configuration
- Output file generation
"""

import os
import sys
import tempfile
import shutil
  1. The convert method signature for MiCoTorchMLIRGen is inferred as convert(output_dir=...). If the real signature differs (e.g., convert(self, path) or returns (path, content)), update the call and assertions accordingly.
  2. The filename suffix _torch_mlir.mlir is assumed from your comment. If the actual implementation uses a different suffix or naming convention, adjust the endswith("_torch_mlir.mlir") assertion to match.
  3. The MiCo overlay assertion currently uses a loose regex for markers like "MiCo", "mico", or "MICO". If the overlay has a specific, known marker (e.g., "// MiCo overlay" or a particular dialect name), replace the assertRegex with a more precise assertIn using that exact marker string.
  4. Ensure that MiCoTorchMLIRGen is already imported in this test module. If not, add the appropriate import at the top of the file (e.g., from mico.torch_mlir.codegen import MiCoTorchMLIRGen) consistent with existing imports and project layout.

Comment on lines +208 to +210
class TestMiCoTorchMLIRGenReset(unittest.TestCase):
"""Test reset functionality."""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Extend reset tests to verify that base-class MLIR state (ops, weights, SSA counters) is also cleared.

Right now the test only verifies the MiCoTorchMLIRGen-specific fields (example_inputs, torch_mlir_ir, mico_quantization_ops). Since MiCoTorchMLIRGen.reset delegates to MiCoMLIRGen.reset, please also assert that the base-class fields are reset (e.g. mlir_ops empty, mlir_weights/mlir_weight_declarations cleared, ssa_counter back to 0). You can populate them via a small forward pass, call reset(), and then check their values to guard against future regressions in the base reset logic.

Suggested change
class TestMiCoTorchMLIRGenReset(unittest.TestCase):
"""Test reset functionality."""
class TestMiCoTorchMLIRGenReset(unittest.TestCase):
"""Test reset functionality."""
def test_reset_clears_base_and_derived_state(self):
# Build a simple model and run a forward pass to populate MLIR state.
model = torch.nn.Sequential(
torch.nn.Conv2d(1, 4, kernel_size=3),
torch.nn.ReLU(),
)
mlir_gen = MiCoTorchMLIRGen(model)
example_input = torch.randn(1, 1, 28, 28)
mlir_gen.forward(example_input)
# Sanity-check that both base-class and derived fields are populated
# before we invoke reset(). This ensures the assertions after reset()
# are meaningful.
self.assertTrue(getattr(mlir_gen, "mlir_ops", []))
self.assertTrue(getattr(mlir_gen, "mlir_weights", {}))
self.assertTrue(getattr(mlir_gen, "mlir_weight_declarations", {}))
self.assertNotEqual(getattr(mlir_gen, "ssa_counter", 0), 0)
self.assertTrue(getattr(mlir_gen, "example_inputs", []))
# torch_mlir_ir might be a string or IR object; just assert it's set.
self.assertIsNotNone(getattr(mlir_gen, "torch_mlir_ir", None))
self.assertTrue(getattr(mlir_gen, "mico_quantization_ops", []))
# Now reset and verify that both base-class and MiCoTorchMLIRGen-specific
# state has been cleared.
mlir_gen.reset()
# Base-class state
self.assertEqual(len(getattr(mlir_gen, "mlir_ops", [])), 0)
self.assertEqual(len(getattr(mlir_gen, "mlir_weights", {})), 0)
self.assertEqual(len(getattr(mlir_gen, "mlir_weight_declarations", {})), 0)
self.assertEqual(getattr(mlir_gen, "ssa_counter", 0), 0)
# Derived-class state
self.assertFalse(getattr(mlir_gen, "example_inputs", []))
self.assertIsNone(getattr(mlir_gen, "torch_mlir_ir", None))
self.assertEqual(len(getattr(mlir_gen, "mico_quantization_ops", [])), 0)

Comment thread doc/MLIR_INTEGRATION.md
```mlir
// Sub-byte integer types
!mico.int<1> // 1-bit (binary)
!mico.int<2> // 2-bit (ternary)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (typo): Clarify the comment for the 2-bit integer type, since "2-bit (ternary)" is inconsistent.

// 2-bit (ternary) is misleading: 2 bits allow 4 values, not 3. Please either change the description or adjust the bit width so the terminology matches.

Suggested change
!mico.int<2> // 2-bit (ternary)
!mico.int<2> // 2-bit (4-level / quaternary)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MLIR Integration

2 participants